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Abstract 

The  U.S.  Department  of  Defense  (DoD)  relies  on  a 
multitude  of  fragmented  simulations  to  assist  in  engineering 
new  systems.  The  DoD  has  recognized  the  need  for  unified 
simulation  environments  to  enhance  the  value  of  new 
models  and  help  achieve  its  defense  transformation  goals;  a 
major  example  of  this  is  the  U.S.  Army's  OneSAF  program. 
However,  no  plan  exists  to  leverage  the  thousands  of 
simulation  models  that  remain  idle  on  shelves.  Localized 
efforts  by  the  government  and  its  contractors  to  unify  such 
models  have  been  marginalized  by  a  number  of  technical 
and  non-technical  hurdles,  some  of  which  are  not  obvious. 
These  include  the  availability  of  models,  the  usability  of 
simulation  construction  tools,  the  creation  of  reference 
architecture,  the  complexity  of  simulation  results,  the 
automation  of  repetitive  integration  tasks,  and  the 
verification  and  validation  of  component  models,  among 
others. 

This  paper  discusses  these  hurdles  in  greater  detail  and 
provides  context  to  DoD  simulation  efforts  from  the  team 
developing  the  Virtual  Systems  Integration  Lab  (VSIL)  for 
the  U.S.  Army  TARDEC.  We  conclude  with 
recommendations  for  establishing  a  unified  approach  to 
maximize  simulation  reuse  across  the  DoD. 

1.  BACKGROUND 

Historically,  the  development  lifecycle  of  most  major 
U.S.  Department  of  Defense  (DoD)  acquisition  and  R&D 
programs  can  be  categorized  as  stovepipe  efforts  that 
operate  independently  of  each  other  and  often  in  parallel. 
As  a  result,  the  DoD  relies  on  a  multitude  of  fragmented 
simulations  to  assist  in  engineering  new  systems. 
Consequently,  many  of  these  simulations  were  duplications 
of  effort.  The  DoD  began  to  recognize  the  value  of  unified 
simulation  environments  to  enhance  the  utility  of  new 
models  and  achieve  its  defense  transformation  goals.  A 
major  example  of  this  was  the  U.S.  Army's  OneSAF 
program,  designed  to  integrate  multi-domain  simulations 
into  one.  The  DoD’s  overall  goal  for  simulation-based 
engineering  is  to  field  the  best  systems  for  the  future 
military  force  in  the  shortest  time  using  the  fewest 
resources. 
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While  the  DoD  recognizes  the  need  to  integrate 
modeling  and  simulation  (M&S)  test  environments  to 
maximize  value  and  achieve  its  goals,  no  plan  exists  to 
leverage  the  thousands  of  existing  simulation  models  that 
remain  idle  in  DoD’s  M&S  portfolio.  Localized  efforts  by 
the  government  and  its  contractors  to  unify  such  models 
have  been  marginalized  by  a  number  of  technical  and  non¬ 
technical  hurdles,  some  of  which  are  not  obvious.  The  first 
step  to  unifying  such  simulations  in  a  useful  manner  is  to 
define  what  that  goal  means. 

2.  DEFECTION  OF  UNIFIED  SIMULATION 

Unified  simulation  is  an  ambitious  goal  for  Systems 
Engineering  that  will  be  reached  once  the  following  criteria 
and  capabilities  are  satisfied  and  delivered: 

•  Interoperability  standards  allow  any  compliant 
simulation  method  to  be  incorporated  (e.g.,  HLA, 
OneSAF) 

•  All  standalone  simulation  models  can  be  integrated 
as  pieces  of  a  bigger  puzzle  (e.g.,  Matlab, 
Simulink,  C++) 

•  A  global  simulation  picture  provides  the  ability  to 
“zoom  in”  on  any  level  of  detail  ranging  from 
systems  to  sub-components 

•  System  design  feedback  gets  generated  that 
accelerates  feasibility  testing  of  hardware  and 
software 

3.  REAL-LIFE  EXAMPLE 

A  real-life  example  of  an  effort  to  unify  systems 
engineering  as  defined  above  is  the  Virtual  System 
Integration  Lab  (VSIL)  program  under  the  U.S.  Army  Tank 
Automotive  Research,  Development  &  Engineering  Center 
(TARDEC).  VSIL  is  a  simulation  suite  for  accelerating 
systems  engineering  that  tests  prototype  designs  prior  to 
committing  to  a  physical  prototype.  Cybernet  was 
contracted  to  develop  the  software  architecture  and  model- 
based  design  methodology  of  the  VSIL. 

3.1.  VSIL  Concept  &  Strategy 

The  VSIL  system  uses  "lego-like”  soft  building  blocks 
that  can  be  dragged  from  a  component  model  library  to 
easily  configure  a  complex  system  for  simulation.  The 
initial  focus  was  on  the  development  of  simulation  models 
that  could  precisely  mimic  the  software  electrical  system 
hardware  of  a  vehicular  platform  (e.g.,  the  electrical  power 
system  architecture  containing  the  wiring  harness,  actuator 
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motors/solenoids,  and  sensors).  The  various  components  of 
the  system  (i.e.,  the  wiring  harness,  actuator 
motors/solenoids,  sensors)  would  be  the  basic  building 
blocks.  These  components  would  model  the  hardware  and 
software  from  the  functional  point  of  view. 

The  goal  was  to  use  these  virtual  components  and 
create  the  complete  electrical  system  for  the  vehicle,  and 
study  its  behavior.  Later,  the  scope  of  the  work  was 
extended  to  other  sub-systems  beyond  the  electrical  system, 
thus  leading  the  way  to  a  complete  virtual  vehicle  build. 

A  key  provision  of  the  VSIL  was  to  allow  users  to  add 
to  the  library  of  modules  with  appropriate  input/output 
definitions,  to  enrich  the  module  library  as  needed.  This 
enabled  a  workflow  for  new  systems  design  that  leveraged 
existing  component  models  to  develop  future  systems,  as 
illustrated  in  Figure  1. 


Figure  1.  VSIL  process  for  new  systems  design 


Currently  there  exist  generic  tools  in  the  market  that  are 
basically  standard  CAE  (computer-aided  engineering) 
software  packages.  These  generic  CAE  tools,  however,  do 
not  offer  the  advantage  of  catering  to  alternative  system 
designs  easily.  They  do  not  use  “soft”  modular  lego-like 
components,  so  that  a  complete  system  or  sub-system  can  be 
quickly  and  easily  “assembled,”  virtually  “built,”  and 
virtually  “run”  by  any  ordinary  user. 

3.2.  VSIL  Objectives 

The  objectives  of  the  VSIL  were  the  following: 

•  Accelerate  and  enhance  next-generation  vehicle 
design  and  development 

•  Increase  efficiency  of  simulation  development 

•  Perform  cost-benefit  analysis  on  component 
models  up  to  full  deployments 


•  Transform  development  process  so  that  new 
vehicle  designs  benefit  from  the  development  of  all 
previous  vehicles 

3.3.  Virtual  Systems  Editor  (ViSE  ) 

Cybernet  has  created  the  capstone  tool  of  the  VSIL 
called  the  Virtual  Systems  Editor,  dubbed  the  “ViSE.”  The 
ViSE  automates  the  creation,  execution,  and  analysis  of 
trade-off  studies.  With  the  ViSE,  the  VSIL  has  achieved  a 
degree  of  unified  simulation,  according  to  our  definition: 

•  The  ViSE  interoperates  with  simulation  models  via 
HLA  and  VSIL  Reference  Architecture  standards. 

•  The  ViSE  leverages  standalone  simulation  models 
for  components  and  subsystems  from  Matlab, 
Simulink,  C/C++,  and  Java. 

•  The  ViSE  enables  the  user  to  observe  the  system- 
level  to  component-level  views  of  the  systems 
being  configured. 

•  The  ViSE  logs  data  from  the  simulation  to  enable 
system  design  analysis  based  on  the  given  scenario. 
This  data  enables  automated  component  trade-off 
analysis  and  requirements  generation. 

Post-analysis  tools  are  in  development  and  are  modeled 
in  Figure  2.  Using  the  ViSE,  a  user  can  experiment  with 
different  configurations  based  on  a  given  scenario  and  a 
library  of  models.  A  screenshot  of  the  ViSE  is  Figure  3. 
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Figure  2.  ViSE  workflow 


Figure  3.  ViSE  software  version  0.5 


4.  HURDLES  TO  UNIFIED  SIMULATION 

The  VSIL  team  encountered  the  following  hurdles 
during  its  joint  simulation  efforts  with  TARDEC: 

•  The  availability  of  models 

•  The  usability  of  simulation  construction  tools 

•  The  creation  of  reference  architecture 

•  The  complexity  of  simulation  results 

•  The  automation  of  repetitive  integration  tasks 

•  The  verification  and  validation  of  component 
models 

4.1.  The  availability  of  models 

The  credibility  of  M&S  is  tied  to  the  availability  and 
fidelity  of  the  component  models  of  interest  (e.g.,  Mobility, 
Suspension).  We  found  that  populating  a  useful  model 
library  from  scratch  was  a  lengthy  task  that  requires  vast 
domain  expertise. 

To  overcome  this  hurdle,  we  added  staff  to  the  VSIL 
team  with  vehicle  systems  expertise  and  sought  out  existing 
models  from  TARDEC. 

4.2.  The  creation  of  Reference  Architecture  (RA) 

RA  defines  the  interfaces  required  by  models  to  be 
leveraged  into  a  unified  simulation  (e.g.,  RA  for  vehicle 
electronics  component  includes  technical  and  non-technical 
attributes  such  as  power  consumption  and  component  cost). 
We  found  that  creating  RA  was  an  exhausting  task.  A 
mature  RA  required  perpetual  re-factoring  over  time. 

To  overcome  this  hurdle,  we  refreshed  the  RA 
iteratively  rather  than  constantly,  after  we  completed  major 
demonstration  milestones. 

4.3.  The  verification  &  validation  of  simulation  models 

True  validation  of  models  was  only  possible  by  using 
real  data  taken  from  the  component  or  system  being 
modeled,  or  by  using  the  most  high-fidelity  models 
available.  However,  those  models  were  not  available  to 
DoD  engineers  or  contractors.  Additionally,  verification 
and  validation  required  definitions  for  “high-fidelity” 
models  and  tiers  of  model  fidelity. 

To  overcome  this  hurdle,  we  established  definitions  and 
criteria  for  judging  model  fidelity  in  documents  pending 
approval. 

4.4.  The  complexity  of  simulation  and  results 

As  systems  were  modeled  and  simulated  at  higher 
fidelity,  the  simulations  produced  progressively  larger  sets 
of  data.  For  example,  swapping  out  five  different  types  of 
components  with  two  models  each  produced  thirty-two  (25) 
iterations  to  execute.  Adding  one  more  type  of  component 
to  trade-off  doubled  the  number  of  iterations  to  sixty-four, 
etc.  This  doubled  the  time  it  took  to  execute  the  simulations 
in  batch.  Using  higher  fidelity  simulations  models  further 
prolonged  the  total  execution  time.  Processing  the  output 
data  sets  in  meaningful  ways  was  also  time-consuming.  We 
needed  better  analysis  tools  to  process  output  data  faster. 

To  overcome  these  hurdles,  we  simplified  and 
shortened  the  test  scenarios  in  a  piecewise  manner.  We  also 


added  provisions  to  end  the  execution  of  scenarios  earlier  if 
key  aspects  of  the  system  failed,  such  as  an  engine  stall. 

4.5.  The  usability  of  simulation  construction  tools 

The  usability  of  tools  impacts  the  efficiency  of  model 
verification  and  validation.  User-friendly  tools  encourage 
more  use,  reduce  anxiety,  and  build  confidence. 

The  typical  users  of  the  VSIL  and  ViSE  were  not 
professional  software  developers,  but  basic  computer  users 
with  mechanical  and  electrical  engineering  backgrounds. 
Other  users  were  managers  who  controlled  budgets  and 
made  acquisition  decisions. 

To  address  the  needs  of  such  users,  we  included  step- 
by-step  instructions  on  running  pre-made  demonstrations 
with  pointers  on  what  to  observe  while  walking  through  the 
software  tools.  We  also  invested  a  significant  amount  of 
time  in  showing  the  tools  to  the  users  in  person,  so  that 
more  questions  could  be  immediately  addressed. 

4.6.  The  automation  of  repetitive  integration  & 
analysis  tasks 

Automating  as  much  of  the  simulation  workflow  as 
possible  was  key  to  achieving  the  benefits  of  unified 
simulation.  The  automated  model  wrapping  for  common 
formats  was  highly  desired.  The  automated  formatting  and 
analysis  of  output  data  was  equally  desired. 

To  address  these  concerns,  we  wrote  technical 
recommendations  for  solving  the  automatic  model-wrapping 
problem  with  various  formats,  including  HLA,  C/C++,  Java, 
and  Matlab/Simulink.  We  also  customized  Perl  scripts  to 
create  trend  plot  overlays  for  each  test  point.  These  scripts 
processed  the  post-simulation  data  in  a  format  more  easily 
consumed  by  Microsoft  Excel.  In  addition,  we  used 
spreadsheet  macros  to  update  a  summary  report  and  ranking 
of  results. 

5.  CONCLUSIONS  FOR  MILITARY  SIMULATION 

Simulation-based  engineering  is  a  vital  but  expensive 
enterprise.  Unified  simulation  is  an  ambitious  goal  that  will 
accelerate  innovation  and  make  systems  engineering  more 
viable  in  the  long  run.  Government  leadership  will  help 
overcome  the  hurdles  to  unifying  military  systems 
engineering  simulations. 

The  DoD  is  the  only  organization  that  can  truly  unify 
systems  engineering  simulations  for  military  use.  Relying 
solely  on  industry  and  non-profits  like  SISO  to  accomplish 
the  task  will  not  achieve  this  goal  in  the  long  term  without  a 
strong  Government  recommendation.  For  this  effort  to  truly 
succeed,  it  will  require  collaboration  among  academia, 
industry,  and  government. 

6.  RECOMMENDATIONS  TO  MAXIMIZE 

SIMULATION  REUSE  ACROSS  DOD 

To  establish  a  unified  approach  to  maximize  simulation 
reuse,  the  DoD  needs  to  strongly  recommend  a  standard 
response  from  industry. 


6.1.  DoD  best  practices  and  provisions 

The  DoD’s  best  practices  for  M&S  must  include  provisions 
for  three  broad  areas: 

1.  Model  Sufficiency 

a.  Are  high-fidelity  models  available? 

b.  Are  they  compliant  with  interoperability 
standards? 

2.  Tool  Usability 

a.  Need  tools  that  highly  automate  the  M&S 
process 

b.  Software  tools  must  be  easy  to  use,  easy 
to  learn,  and  fast 

3.  Process  Adoption 

a.  Need  usage  to  get  credibility  and 
continuous  improvement 

b.  Write  model  deliverables  into  contracts 

c.  Make  model  repositories  easily  searchable 

6.2.  Recommend  wider  deployments  of  existing  efforts 
The  adoption  of  simulation-based  processes  and 

toolsets  in  the  defense  space  will  gain  the  most  traction 
when  recommended  with  ongoing  efforts. 

For  example,  existing  programs  such  as  OneSAF 
should  publish  their  plan  on  how  they  will  interoperate  with 
new  models.  The  next  evolution  of  OneSAF  should 
incorporate  higher  fidelity  simulations  of  FCS  models, 
which  may  already  exist. 

Since  OneSAF  is  expected  to  be  a  platform  for  other 
services  if  it  continues  to  be  successful,  this  should  trigger  a 
number  of  action  items  including:  discovering  needed 
models,  identifying  interoperability  protocols,  and  designing 
necessary  extensions  to  incorporate  OneSAF  into  new 
programs. 

6.3.  Employ  a  bottom-up  approach  to  unifying 
simulations. 

Experience  shows  that  a  bottom-up  approach  to 
unifying  simulations  is  superior  to  a  top-down  approach. 
For  example,  the  expansive  JSIM  project  that  preceded 
OneSAF  failed  due  to  the  management  burdens  of  operating 
as  a  joint-service  project. 

6.4.  Account  for  ongoing  simulation  interoperability 
efforts. 

A  unified  approach  relies  on  simulation 
interoperability.  The  DoD  should  consider  how  ongoing 
infrastructure  developments  in  the  DoD  community  will  fit 
in.  These  efforts  include  F1LA,  BOMS,  SEDRIS,  and 
MSDL  (Military  Scenario  Definition  Language). 

The  products  from  these  community  efforts  include 
new  standards  and  conventions  for  adapting  modern 
simulation  methods,  as  well  as  adapting  legacy  models. 


6.5.  Populate  government-owned  model  repositories. 
Let  industry  maintain  proprietary  repositories 
with  interface-based  model  access. 

The  principle  of  “garbage-in,  garbage-out”  dictates  that 
achieving  highly  accurate  simulations  requires  access  to 
more  accurate  data.  When  access  to  existing  high-fidelity 
models  is  restricted  or  not  available  to  Government 
engineers  and  contractors,  the  overall  quality  of  simulations 
in  the  DoD  community  suffers. 

This  recommendation  gives  both  the  DoD  and  industry 
access  to  models  while  protecting  their  intellectual  property. 
The  DoD  would  allow  contractors  to  interface  with 
decentralized  repositories  based  on  service  contracts,  similar 
to  the  way  that  DMSO  sponsors  M&S  efforts  such  as 
SEDRIS.  This  policy  will  provide  Government  engineers 
and  support  contractors  more  accurate  models  and  data  to 
use. 

6.6.  Establish  a  validation  program  for  simulation 
models. 

A  validation  program  is  necessary  to  verify  the 
adequacy  of  simulation  models.  This  program  can  be  run  by 
a  university  center,  the  way  Johns  Hopkins  was  contracted 
to  perform  HLA  RTI  compliance  testing  for  DMSO. 

6.7.  Invest  in  a  standard  simulation-based  design 
environment. 

Investing  in  a  standard  simulation-based  design 
environment  will  enable  the  DoD  to  send  a  tangible 
recommendation  to  its  PEOs  and  contractors.  DoD  will 
want  to  identify  a  software  toolset  that  is  easy  to  use, 
accurate,  useful,  and  flexible. 

6.8.  Require  the  delivery  of  component  models 
developed  under  contract. 

To  execute  this  recommendation,  the  Government  and 
its  contractors  will  need  standardized  tools  to  handoff  and 
evaluate  models.  The  DoD  will  want  more  automated  M&S 
capabilities  to  get  more  ROI,  and  should  survey  the  market 
for  better  tools  to  effectively  manage  M&S. 
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